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SUBJECT:  Audit  Report  on  Engine  Monitoring  Systems  for  Jet  Aircraft  Engines 
(Report  No.  96-195) 


We  are  providing  this  report  for  your  review  and  comment.  Management 
comments  on  a  draft  of  this  report  were  considered  in  preparing  the  final  report. 


DoD  Directive  7650.3  requires  that  all  recommendations  be  resolved  promptly. 
The  Deputy  Under  Secretary  of  Defense  for  Logistics  comments  were  responsive  and 
additional  comments  are  not  required.  The  Army  concurred  or  partially  concurred  with 
all  recommendations,  but  its  comments  on  Recommendation  2.  and  Recommendation  4. 
did  not  fully  address  the  issues.  The  Navy  and  the  Air  Force  did  not  respond  to  the 
draft  of  this  report.  We  request  that  the  Army,  the  Navy,  and  the  Air  Force  provide 
comments  on  the  unresolved  recommendations  by  August  19,  1996. 


We  appreciate  the  courtesies  extended  to  the  audit  staff.  Questions  on  the  audit 
should  be  directed  to  Mr.  John  A.  Gannon,  Audit  Program  Director,  at  (703)  604-9427 
(DSN  664-9427)  or  Mr.  Joseph  M.  Austin,  Audit  Project  Manager,  at  (703)  604-9424 
(DSN  664-9424).  Copies  of  the  report  will  be  distributed  to  the  organizations  listed  in 
Appendix  E.  The  audit  team  members  are  listed  on  the  inside  back  cover. 


Robert  J.  Lieberman 
Assistant  Inspector  General 
for  Auditing 
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Engine  Monitoring  Systems  for  Jet  Aircraft  Engines 


Executive  Summary 


Introduction.  An  engine  monitoring  system  (EMS)  is  a  maintenance  tool  used  to 
determine  the  condition  of  jet  aircraft  engines  as  they  accumulate  hours  of  operation. 
The  purpose  of  engine  monitoring  is  to  assist  in  identifying  and  correcting  problems 
before  they  result  in  engine  failures.  Specifically,  an  EMS  is  designed  to  detect 
malfunctions  during  normal  aircraft  operations  (event  detection)  and  to  gather 
information  for  engine  performance  analysis  during  take  off  and  flight  operations  (trend 
recording). 

Audit  Objectives.  The  objectives  were  to  evaluate  the  effectiveness  of  DoD  systems 
being  developed  and  systems  being  used  to  monitor  the  conditions  of  jet  aircraft 
engines,  including  the  ability  of  the  monitoring  systems  to  provide  accurate  and  reliable 
engine  performance  data  about  the  condition  of  the  engines,  and  the  cost-effectiveness 
of  the  systems.  We  also  evaluated  the  overall  management  of  the  systems.  In  addition, 
we  evaluated  management  controls  applicable  to  the  audit  objectives. 

Audit  Results.  The  Military  Departments  needed  to  improve  their  EMS  programs. 
Specifically,  existing  systems  do  not  provide  all  the  information  needed  to  diagnose, 
monitor,  and  trend  the  condition  of  the  engines.  With  effective  oversight,  systems 
being  developed  will  correct  some  of  the  identified  deficiencies.  Additionally,  the 
Army  and  the  Navy  centralized  collection  of  EMS  data  was  not  consistent  and  timely, 
and  the  Military  Departments  lacked  performance  measures  for  determining  the  cost 
and  operational  effectiveness  of  EMS.  Further,  updating  technical  data  in  the 
Air  Force  and  EMS  training  in  the  Military  Departments  needed  improvement.  As  a 
result,  existing  EMS  programs  do  not  ensure  that  maintenance  personnel  are  provided 
with  the  necessary  data  and  trained  on  the  systems  to  improve  the  efficiency  of 
maintenance  on  jet  aircraft  engines.  See  Part  I  for  a  discussion  of  the  audit  results. 

The  audit  identified  material  management  control  weaknesses  in  that  controls  were  not 
in  place  to  ensure  that  existing  EMS  programs  provided  maintenance  personnel  with  the 
necessary  data  and  training  on  the  systems  to  improve  the  efficiency  of  maintenance  on 
jet  aircraft  engines.  See  Appendix  A  for  management  controls  assessed  and  for  a 
discussion  of  material  weaknesses  identified. 

Summary  of  Recommendations.  We  recommend  that  the  Deputy  Under  Secretary  of 
Defense  for  Logistics  provide  management  oversight  over  the  collection,  processing, 
recording,  and  use  of  data  that  the  proposed  Joint  Advanced  Health  and  Usage 


Monitoring  System  generated.  We  also  recommend  that  the  Army,  the  Navy,  and  the 
Air  Force  issue  guidance  to  require  performance  specifications  for  EMS  equipment 
under  development  and  guidance  to  establish  performance  measures.  Additionally,  we 
recommend  that  die  Army  and  the  Navy  selectively  test  the  timeliness  of  EMS  data 
reported  by  maintenance  organizations  and  provide  additional  training  where  necessary 
to  maintenance  personnel  on  timely  reporting  of  EMS  data.  Further,  we  recommend 
that  the  Navy  provide  additional  training  to  maintenance  personnel  on  the  operation  of 
systems  used  to  collect  and  report  EMS  data.  We  also  recommend  that  the  Air  Force 
initiate  a  review  of  the  EMS  technical  order  change  process,  determine  EMS  training 
requirements,  establish  follow-on  training,  and  make  the  required  training  available  to 
EMS  users. 

Management  Comments.  The  Deputy  Under  Secretary  of  Defense  for  Logistics 
concurred  with  our  recommendation  to  provide  management  oversight  on  the  Joint 
Advanced  Health  and  Usage  Monitoring  System  program.  The  Deputy  Under 
Secretary  of  Defense  for  Logistics  stated  that,  in  the  case  of  the  Joint  Advanced  Health 
and  Usage  Monitoring  System,  the  Deputy  Under  Secretary  of  Defense  for  Logistics 
will  provide  management  oversight  in  coordination  with  the  Deputy  Under  Secretary  of 
Defense  (Advanced  Technology).  The  Army  concurred  with  the  recommendation  to 
issue  guidance  requiring  that  performance  specifications  be  developed  for  all  EMS 
under  development.  The  Army  also  concurred  with  the  recommendation  to  selectively 
test  the  consistency  and  timeliness  of  EMS  data  being  reported  by  maintenance 
organizations.  The  Army  partially  concurred  with  the  recommendation  to  develop 
guidance  to  establish  useful  performance  measures  to  determine  the  cost  and  operational 
effectiveness  of  EMS  programs  already  in  existence  and  those  under  development.  The 
Army  stated  that  EMS  programs  under  development  are  being  developed  to 
performance  specifications  that  will  ensure  that  the  appropriate  data  are  generated  to 
monitor  engine  condition,  diagnose  faults,  and  trend  engine  performance.  The  Army 
also  stated  that  action  had  been  initiated  to  improve  data  reporting  for  the  T-700 
engine.  Additionally,  the  Army  stated  that  analyses  will  be  performed  on  the  T-800 
engine,  currently  being  developed,  to  assess  the  value  added  by  the  diagnostic  system. 
However,  the  response  stated  that  an  analysis  on  the  T-700  engine  is  not  feasible 
because  it  does  not  have  an  EMS  installed  on  it.  See  Part  I  for  a  discussion  of 
management  comments  and  Part  III  for  the  complete  text  of  the  management 
comments. 

The  Navy  and  the  Air  Force  did  not  comment  on  a  draft  of  this  report. 

Audit  Response.  Comments  from  the  Deputy  Under  Secretary  of  Defense  for 
Logistics  were  responsive.  The  Army's  comments  were  not  fully  responsive  because 
the  comments  did  not  address  the  need  to  issue  guidance  requiring  performance 
specifications  for  current  and  future  EMS  developments  and  the  need  to  develop 
guidance  to  establish  useful  performance  measures  for  determining  the  cost  and 
operational  effectiveness  of  EMS.  Additionally,  the  Army  did  not  indicate  a 
completion  date  for  each  action  to  be  taken.  We  request  that  the  Army,  the  Navy,  and 
the  Air  Force  provide  comments  on  the  final  report  by  August  19,  1996. 
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Part  I  -  Audit  Results 


Audit  Results 


Audit  Background 


An  engine  monitoring  system  (EMS)  is  a  maintenance  tool  used  to  determine  the 
condition  of  jet  aircraft  engines  as  they  accumulate  hours  of  operation.  The 
purpose  of  engine  monitoring  is  to  assist  in  identifying  and  correcting  engine 
problems  before  they  result  in  engine  failures.  Specifically,  an  EMS  is  designed 
to  detect  malfunctions  during  normal  aircraft  operations  (event  detection)  and  to 
gather  information  for  engine  performance  analysis  during  take  off  and  flight 
operations  (trend  recording).  Data  that  EMS  collect  are  used  to  isolate  faulty 
engine  components  and  systems,  forecast  engine  deterioration  and  failure,  and 
predict  the  life  remaining  on  engines  and  their  components. ' 

Technologically,  engine  monitoring  of  jet  aircraft  is  an  evolving  concept  and  a 
single,  standard  EMS  has  not  been  developed.  As  a  result,  each  EMS  that  DoD 
acquires  and  operates  is  unique  to  a  specific  jet  aircraft  engine.  Further,  EMS 
hardware  used  by  DoD  ranges  from  a  simple  history  recorder  that  registers 
hours  of  operation  to  very  complex  equipment  that  generates  diagnostic  trouble 
codes  and  records  internal  engine  performance  data.  Generally,  an  EMS 
consists  of  an  engine-mounted  recorder  that  uses  sensors  to  gather  data  on 
specified  performance  characteristics,  and  a  ground  system  that  processes  and 
stores  the  data.  Data  gathered  by  the  recorder  are  transferred  to  a  ground 
computer  system  using  a  downloading  device,  such  as  a  laptop  computer, 
hand-held  scanner,  or  floppy  disk. 

EMS  Evaluated.  During  the  audit,  we  evaluated  the  history  recorder  installed 
on  the  Army  Apache  and  Blackhawk  helicopter  engines.  We  also  evaluated 
more  complex  EMS,  which  were  installed  on  the  Navy  F-14B/D  and 
F/A-18A/D  aircraft  engines,  and  on  the  Air  Force  B-1B,  F-15E,  and  F-16C/D 
aircraft  engines. 

Maintenance  Concepts  and  Objectives.  Each  EMS  employed  has,  as  its 
objective,  the  support  of  the  maintenance  concept  established  for  that  engine. 
The  maintenance  support  concept  of  aircraft  engines  has  typically  included  three 
levels  of  maintenance:  organizational  (on-aircraft),  intermediate  (base 
maintenance  shops),  and  depot  (complete  disassembly,  repair,  modification 
incorporation,  reassembly,  and  performance  tests).  Base  maintenance  objectives 
were  to  identify  the  malfunctioning  module  or  accessory  with  a  high  degree  of 
assurance  to  minimize  faults  that  cannot  be  corroborated  when  engine  modules 
or  assemblies  were  returned  to  depot.  The  requirement  for  fault  isolation  and 
early  detection  of  malfunctions  established  the  need  for  EMS.  The  maintenance 
concept  of  an  engine  must  consider  maintenance  actions  that  can  generate 
scheduled  and  unscheduled  engine  removals. 
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Audit  Results 


The  maintenance  concept  for  an  engine  with  components  that  have  limited  life  is 
driven  by  the  percent  of  life  remaining  for  those  components.  The  integrity  of 
the  recording  of  life  cycle  fatigue  and  hour  life  consumption  is  the  basis  for  the 
engine  maintenance  concept.  The  maintenance  concept  for  an  engine  with 
trending  capabilities  is  driven  by  the  ability  to  record  and  analyze  shifts  (trends) 
in  the  performance  of  the  engine.  Monitoring  and  selective  documentation  of 
performance  shifts  can  provide  information  to  be  used  for  making  maintenance 
decisions.  Engine  trending  is  a  maintenance  process  of  structured  analysis  of 
operational  and  historical  aircraft  engine  data  to  provide  insight  to  forecast 
engine  component  failures  and  deterioration  using  the  engine's  own  signature 
characteristics  and  performance  data.  Trending  includes  monitoring  the 
performance  of  engines  and  identifying  limits  of  operating  parameters. 
Trending  supports  scheduled  maintenance  performed  on  the  engine  because  it 
defines  performance  degradation.  Trending  also  produces  unscheduled 
maintenance  requirements  because  trend  alarms  are  generated  when 
predetermined  parameter  thresholds  are  approached  or  exceeded.  See 
Appendix  C  for  a  description  of  the  mechanics  of  EMS. 


Audit  Objectives 


The  primary  audit  objectives  were  to  evaluate  the  effectiveness  of  DoD  systems 
being  developed  and  used  to  monitor  the  condition  of  jet  aircraft  engines, 
including  the  ability  of  the  systems  to  provide  accurate  and  reliable  engine 
performance  data  about  the  condition  of  die  engines,  and  the  cost-effectiveness 
of  the  systems.  We  also  evaluated  the  overall  management  of  the  systems.  In 
addition,  we  evaluated  the  adequacy  of  the  Military  Departments'  management 
control  programs  applicable  to  the  audit  objectives.  See  Appendix  A  for  a 
discussion  of  the  scope,  methodology  and  management  control  program 
coverage,  and  Appendix  B  for  a  summary  of  prior  audit  coverage  related  to  the 
audit  objectives. 
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Management  of  Engine  Monitoring 
Systems  Programs 

The  Military  Departments  needed  to  improve  their  EMS  programs. 
Specifically,  existing  EMSs  did  not  provide  all  the  information  needed  to 
diagnose,  monitor,  and  trend  the  condition  of  the  engines.  With 
effective  oversight  by  the  Deputy  Under  Secretary  of  Defense  for 
Logistics  and  the  Military  Departments,  the  EMS  being  developed  will 
correct  some  of  the  identified  deficiencies.  Additionally,  the  Army  and 
the  Navy  centralized  collecting  and  reporting  of  EMS  data  were  not 
consistent  and  timely,  and  the  Military  Departments  lacked  performance 
measures  for  determining  the  cost  and  operational  effectiveness  of  EMS. 
Further,  updating  technical  data  in  the  Air  Force  and  EMS  training  in 
the  Military  Departments  needed  improvement.  Those  conditions  were 
attributed  to  inadequate  guidance  on  performance  specifications  and  to 
inadequate  management  oversight.  As  a  result,  existing  EMS  programs 
did  not  ensure  that  maintenance  personnel  were  provided  with  the 
necessary  data  and  trained  on  the  systems  to  improve  the  efficiency  of 
maintenance  on  jet  aircraft  engines. 


Policy  and  Guidance  for  Engine  Maintenance  Programs 


DoD  Directive  4151.18,  "Maintenance  of  Military  Materiel,"  August  12,  1992, 
establishes  policies  and  assigns  responsibilities  for  performance  of  DoD  materiel 
maintenance  at  all  levels  (organizational,  intermediate,  and  depot)  and  for  both 
organic  and  contractor  organizations.  The  Directive  requires  DoD  Components 
to  provide  an  adequate  program  for  the  maintenance  of  assigned  materiel. 

Department  of  the  Army  Technical  Bulletin  1-2840-248-20-18,  "Life-limits  for 
T700-GE-700  and  T700-GE-701  Engine  Components,"  provides  instructions  for 
determining  and  recording  the  life-limits  for  gas  generator  turbine  rotor 
components.  The  technical  bulletin  states  that  an  engine  history  recorder  will  be 
used  for  tracking  life-limited  components.  It  assigns  the  maintenance  units  the 
responsibility  of  ensuring  that  component  life-limits  are  not  exceeded. 

Department  of  the  Army  Pamphlet  738-751,  "Functional  Users  Manual  for  the 
Army  Maintenance  Management  System- Aviation, "  June  15,  1992,  prescribes 
procedures  for  collection  and  submission  of  history  recorder  data  to  die  Aviation 
and  Troop  Command  (ATCOM).  The  pamphlet  states  that  the  data  should  be 
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collected  monthly  and  recorded  on  the  Engine  Operating  Homs  Record 
(DA  Form  2408-19-3),  then  submitted  to  ATCOM  for  updating  the  central  data 
base  used  for  storing  historical  engine  data. 

The  Naval  Air  Systems  Command  Instruction  4790.21,  "Flight  Information 
Recording  and  Monitoring  Systems,"  October  4,  1989,  states  that  a  standardized 
intermediate  level  software  package  for  a  common  intermediate  level  ground 
station  will  be  developed  that  will  interface  with  the  organizational  level  aircraft 
data  systems  and  allow  for  reporting  and  storing  of  historical  data.  The  Naval 
Aviation  Logistics  Data  Analysis  (NALDA)  system  will  be  used  for  reporting 
and  storing  of  historical  data.  Where  practical,  the  intermediate  level  and 
organizational  level  organizations  will  use  common  hardware. 

Commander  Naval  Air  Atlantic  Instruction  4790.38,  "Enhanced  Comprehensive 
Asset  Management  System,"  October  25,  1990,  and  Commander  Naval  Air 
Pacific  Instruction  4790.33,  "Enhanced  Comprehensive  Asset  Management 
System,"  April  15,  1988,  state  that  intermediate  level  maintenance  organizations 
will  transmit  all  parts  life  tracking  system  data  through  electronic  networking  to 
the  NALDA  central  data  base  daily. 

Air  Force  Instruction  21-104,  "Selective  Management  of  Selected  Gas  Turbine 
Engines,"  June  17,  1994,  attachment  2,  "Engine  Management,"  states  that 
engine  trending  and  diagnostics  must  be  used  for  all  engines.  The  Instruction 
establishes  the  Comprehensive  Engine  Management  System  (CEMS)  IV  as  the 
Air  Force  standard  system  for  processing,  correlating,  and  plotting  engine 
performance,  oil  analyses,  and  maintenance  data  used  to  diagnose  the  condition 
of  engines. 


Effective  EMS  Programs 


Public  and  private  sector  experts  agree  that,  for  an  EMS  to  be  effective,  it  must 
perform  certain  functions,  including  real-time  detection  of  malfunctions, 
recording  and  storing  of  trend  data,  recording  parts  life  tracking  data,  and 
interfacing  with  aircraft  ground  equipment  to  transfer  collected  data.  The 
ability  of  an  EMS  to  accomplish  diagnostic  functions  includes  the  capability  to 
record  fault  information,  present  advisory  information  to  the  maintenance 
technician,  and  document  performance  parameters  after  corrective  maintenance 
actions  have  been  taken. 
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Existing  and  Planned  EMS  Development 


Military  Departments  needed  to  improve  their  EMS  programs  because  the 
existing  programs  did  not  provide  all  the  information  needed  to  diagnose, 
monitor,  and  trend  the  condition  of  engines.  The  condition  existed  because  of 
inadequate  guidance  and  management  oversight.  With  effective  oversight,  new 
systems  being  developed  will  correct  some  of  the  deficiencies. 

Existing  EMS  Programs.  The  EMS  installed  on  military  aircraft  included  in 
our  review  did  not  provide  the  information  needed  to  better  evaluate  the 
condition  of  engines. 

Army.  The  history  recorder  installed  on  the  engine  of  the  Army  Apache 
and  Blackhawk  helicopters  was  designed  to  provide  parts  life  tracking,  but  not 
diagnostic  and  trending  capabilities.  The  recorder  did  not  provide  data 
concerning  the  condition  of  the  engine  and  did  not  provide  maintenance 
personnel  with  tools  to  assist  in  troubleshooting  problems  associated  with  engine 
or  engine  components.  Further,  the  recorder  had  no  in-flight  data  collection 
and  analysis  capability.  It  measured  only  mechanical  and  thermal  stress,  time 
temperature  index,  and  actual  engine  operating  hours.  The  number  of  operating 
hours  that  the  recorder  accumulated  were  used  to  compute  the  remaining  life  of 
specific  components.  In  addition,  the  Army  had  no  ground  computer  system  to 
capture  and  process  the  data  recorded  during  flight.  When  the  Army  acquired 
the  engines  in  1978,  Army  guidance  did  not  define  development  standards  for 
the  systems  and  did  not  address  requirements  for  diagnostics  and  trending. 

Navy.  The  Navy  in-flight  EMS  records  engine  status,  tracks  engine  use, 
records  pre-event  and  post-event  engine  data  that  alert  the  pilot  during  flight  to 
serious  engine  anomalies,  and  provides  maintenance  codes  for  the  ground  crew. 
Navy  ground-based  computer  systems  download,  process,  and  store  engine 
performance  data  recorded  during  flight.  They  also  produce  maintenance 
reports  to  be  used  for  troubleshooting  and  calculate  and  store  engine  data  for 
parts  life  tracking.  Because  of  inadequate  guidance  and  oversight,  the  Navy 
ground  computer  systems,  however,  lacked  trending  capabilities.  The  Navy 
also  did  not  establish  a  standard  ground  computer  system  for  processing  in-flight 
EMS  data.  For  example,  the  F-14B/D  and  F/A-18A/D  aircraft  had  different 
ground  computer  systems.  The  F-14B/D  aircraft  used  the  preproduction 
maintenance  data  processing  system,  while  the  F/A-18A/D  aircraft  used  the 
enhanced  comprehensive  asset  management  system  (ECAMS).  Neither  ground 
system  was  designed  to  produce  trending  data  to  predict  imminent  engine  failure 
or  to  allow  enhanced  scheduling  of  maintenance  actions. 

Air  Force.  The  Air  Force  EMS  provided  full  diagnostics  and  parts  life 
tracking  capabilities  and  limited  trending  capabilities.  The  Air  Force  B-1B, 
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F-15E,  and  F-16C/D  aircraft  each  used  a  different  in-flight  EMS.  Although  the 
three  in-flight  EMS  had  full  diagnostic  and  parts  life  tracking  capabilities,  none 
had  full  trending  capabilities,  either  because  of  limitations  of  the  in-flight  EMS 
or  because  of  the  ground  processing  system.  For  example,  the  F-15E  in-flight 
EMS  recorded  only  eight  raw  data  elements  that  could  be  used  for  trending. 
The  limited  raw  data  elements  that  the  EMS  recorded  permitted  trending  only 
for  warranted  performance  measures,  and  the  Air  Force  could  not  use  the  data 
for  maintenance  purposes.  In  contrast,  the  in-flight  EMS  on  the  F-16C/D 
aircraft  recorded  48  raw  data  elements  that  were  used  for  trending.  As  a  result, 
the  F-16C/D  in-flight  EMS  provided  Air  Force  maintenance  personnel  with 
trendable  data  for  three  types  of  engine  malfunctions,  while  the  F-15E  did  not 
provide  any  trending  capabilities  that  were  useful  for  maintenance  purposes. 
The  difference  in  EMS  capability  resulted  from  the  Air  Force  acquiring  engines 
without  developing  EMS  performance  specifications  for  trending.  For  example, 
the  original  specifications  for  the  F-15E  EMS  were  very  general  and  required 
only  that  the  system  should  detect  and  isolate  faults.  The  Air  Force  did  not 
incorporate  a  requirement  for  trending  in  the  engine  or  EMS  performance 
specifications.  Therefore,  Air  Force  guidance  and  management  oversight  for 
the  EMS  program  was  insufficient  to  ensure  that  engines  were  procured  with 
EMS  capable  of  performing  trending. 

Additionally,  while  the  EMS  on  the  B-1B  and  F-16C/D  aircraft  recorded 
enough  data  for  trending,  CEMS  IV  could  not  fully  process  and  analyze  the  data 
to  trend  for  more  than  three  types  of  engine  malfunctions.  The  CEMS  IV 
limitation  occurred  because  the  Air  Force  did  not  provide  the  CEMS  IV 
program  office  with  trend  signatures  (trend  profiles)  of  impending  failures  for 
the  B-1B,  F-15E,  and  F-16C/D  aircraft  engines.  CEMS  IV  had  the  capability 
to  pattern  match  the  trend  signatures;  however,  the  signatures  needed  to  first  be 
identified  and  made  available. 

Planned  EMS  Development.  The  EMSs  planned  for  the  military  aircraft  will 
improve  the  capability  of  EMS  programs  if  the  Deputy  Under  Secretary  of 
Defense  for  Logistics  and  the  Military  Departments  ensure  adequate 
management  oversight. 

Deputy  Under  Secretary  of  Defense  for  Logistics.  On  March  19, 
1996,  the  Naval  Air  Systems  Command  briefed  the  Deputy  Under  Secretary  of 
Defense  for  Logistics  on  the  proposed  Joint  Advanced  Health  and  Usage 
Monitoring  System.  The  proposed  system  is  an  advanced  concept  technology 
demonstration  that  the  Army  and  the  Navy  will  jointly  develop  for  installation 
and  testing  on  six  Army  and  six  Navy  helicopters.  The  system  is  designed  to 
detect  and  predict  both  dynamic  component  and  structural  failures  before  the 
failures  occur.  The  system  will  monitor  the  helicopter  drive,  engine,  rotor,  and 
structure  subsystems.  The  objectives  of  the  system  are  to  improve  aircraft 
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safety  and  reliability,  increase  operational  availability,  reduce  life-cycle  costs, 
and  streamline  maintenance  and  logistics  infrastructure.  If  approved,  the  system 
is  scheduled  to  be  developed  over  a  4-year  period  (1997  through  2000)  at  a  cost 
of  about  $16  million. 

The  joint  program,  as  presented,  shows  a  great  deal  of  promise.  For  the 
program  to  be  successful,  the  Deputy  Under  Secretary  of  Defense  for  Logistics 
needs  to  provide  management  oversight  of  its  design,  manufacture,  and 
implementation.  The  Deputy  Under  Secretary  of  Defense  for  Logistics  agreed 
with  the  need  for  a  cost  benefit  analysis  and  the  need  for  performance 
specifications  and  standards  for  measuring  cost  and  operational  effectiveness. 
However,  the  Joint  Advanced  Health  and  Usage  Monitoring  System  will 
generate  large  amounts  of  data;  therefore,  adequate  collecting,  processing, 
recording,  and  using  the  data  are  crucial.  As  a  result,  the  Deputy  Under 
Secretary  of  Defense  for  Logistics  should  work  with  the  Military  Departments 
to  provide  adequate  management  oversight  of  those  functions. 

Army.  In  1993,  the  Army  contracted  with  General  Electric  to  develop 
the  Turbine  Engine  Diagnostic  System  to  replace  the  history  recorder  on  future 
helicopter  engine  buys.  The  objective  of  the  program  was  to  develop  improved 
diagnostic  and  fault  isolation  capabilities  that  would  result  in  reduced  false 
engine  removals,  low  maintenance  time,  and  decreased  operating  costs. 
However,  the  Turbine  Engine  Diagnostic  System  will  not  have  die  capability  to 
perform  trending.  General  Electric  gathered  and  analyzed  data  from 
maintenance  units  and  identified  areas  where  improvements  could  be  made. 
Areas  identified  included  changes  in  on-board  recorders,  test  equipment, 
troubleshooting  manuals,  and  training.  The  Army  suspended  the  development 
effort  in  November  1995  until  the  Army  completes  a  cost  benefit  analysis. 

Navy.  In  1991,  the  Navy  started  using  the  preproduction  maintenance 
data  processing  system  for  processing  F-14B/D  in-flight  EMS  data.  In  August 
1995,  the  Navy  replaced  the  preproduction  maintenance  data  processing  system 
with  the  new  maintenance  data  processing  system.  The  new  maintenance  data 
processing  system  was  designed  to  support  the  on-condition  (change  components 
only  when  necessary)  maintenance  concept  for  the  engines  and  will  be  the 
standard  ground  computer  system  that  will  support  F-14B/D,  F/A-18A/D,  and 
V-22  aircraft  maintenance  data  processing  requirements  at  the  organizational, 
intermediate,  and  depot  levels  of  maintenance.  In  addition  to  diagnostic 
capability  and  tracking  of  components  with  limited  life,  the  system  will  provide 
trending  capability.  In  September  1995,  the  Navy  discontinued  use  of  die  new 
maintenance  data  processing  system  because  the  experienced  personnel  who 
developed  the  system  retired  or  left  the  Navy  when  they  learned  that  the  Naval 
Air  Warfare  Center  in  Indianapolis,  Indiana,  was  scheduled  to  close  because  of 
downsizing.  The  Navy  is  resolving  that  problem  by  hiring  new  data  processing 
specialists. 
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Air  Force.  The  Air  Force  used  CEMS  IV  to  collect  and  analyze 
EMS-generated  data  for  the  B-1B,  F-15E,  and  F-16C/D  aircraft  engines,  as  well 
as  for  six  additional  types  of  aircraft  engines.  However,  CEMS  IV  provided 
only  limited  trending  capabilities.  For  example,  CEMS  IV  automatically 
generated  only  three  trend  caution  codes  for  all  of  the  possible  malfunctions  on 
the  F-16C/D  aircraft  engine,  compared  to  the  80  diagnostic  fault  codes  that  the 
system  generated.  To  correct  that  deficiency,  the  Air  Force  developed  a 
software  enhancement  to  CEMS  IV  that  was  intended  to  further  optimize 
trending  capabilities.  The  new  software,  intelligent  trending  and  diagnostic 
system,  was  designed  to  operate  in  concert  with  CEMS  IV  to  automatically 
perform  complex  analyses  of  raw  data.  Its  trend  analysis  and  processing 
capabilities  should  identify  potential  problems  and  produce  automatic  codes  that 
will  alert  maintenance  personnel  of  impending  engine  malfunctions.  The 
software  enhancement  will  initially  allow  trending  for  12  more  types  of  engine 
malfunctions  than  CEMS  IV  alone  provides  for  the  F-16C/D  aircraft  engine.  In 
its  intelligent  trending  and  diagnostic  system  implementation,  the  Air  Force 
plans  to  develop  intelligent  trending  and  diagnostic  system  modules  for  four 
additional  types  of  engines  to  be  used  on  six  types  of  aircraft. 

The  analytical  capability  that  the  intelligent  trending  and  diagnostic  system 
added  will  provide  little  or  no  trending  capability  for  aircraft  with  EMS  that 
record  only  a  few  trendable  parameters.  The  only  way  to  expand  the  potential 
of  the  in-flight  EMS  for  the  F-15E  aircraft,  for  example,  would  be  to  upgrade 
the  in-flight  EMS  hardware  or  software.  Both  the  Air  Force  and  Pratt  and 
Whitney,  the  engine  manufacturer,  contended  that  an  upgrade  to  the  F-15E 
aircraft  EMS  would  have  little  or  no  impact  on  engine  maintenance.  However, 
Pratt  and  Whitney  assessed  the  operational  impact  of  only  one  type  of  potential 
upgrade.  It  evaluated  3  years  of  field-level  trending  and  performance  data 
under  a  component  improvement  task.  The  task  focused  on  the  development  of 
two  algorithms  for  using  take-off  data  to  trend  engine  performance.  The  study 
concluded  that  performance  trending  of  engine  diagnostic  take-off  data  would 
not  enhance  the  maintenance  process  for  the  F-15E  engine.  In  1994,  the 
Air  Force  discontinued  funding  for  the  task.  However,  the  component 
improvement  task  was  a  review  of  only  one  alternative  for  providing  trend 
capabilities.  Neither  the  Air  Force  nor  Pratt  and  Whitney  conducted  a  cost 
benefit  analysis  of  other  trend  alternatives,  such  as  those  identified  in  the  July 
1991  Pratt  and  Whitney  study,  "Investigation  of  Trending  Applications  for  the 
F-100  Engine  Using  the  CEMS  IV  Computer  System."  Therefore,  practical 
trend  applications  for  the  F-15E  engine  had  not  been  fully  explored.  As  a 
result,  the  Air  Force  did  not  definitely  know  the  impact  that  upgrading  the 
F-15E  EMS  for  trending  would  have  on  establishing  cost-effective  trend 
analyses  and  consequently  on  improving  maintenance  capabilities. 
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EMS  Data  Collection  and  Performance  Measures 


Collection  and  Reporting  of  EMS  Data.  The  Army  and  the  Navy  centralized 
collecting  and  reporting  of  EMS  data  were  not  consistent  and  timely.  However, 
the  Air  Force  centralized  collecting  and  reporting  of  data  were  well  managed. 

Army.  Within  the  Army,  ATCOM  is  responsible  for  the  management 
and  oversight  of  the  EMS  program.  Department  of  the  Army  Pamphlet  738-751 
states  that  maintenance  personnel  at  intermediate  and  organizational  level 
maintenance  organizations  should  collect  the  history  recorder  data  monthly  and 
record  the  data  on  the  Engine  Operating  Homs  Record  (DA  Form  2408-19-3). 
Maintenance  personnel  should  provide  the  data  to  ATCOM  to  update  the  central 
data  base  that  is  used  for  storing  historical  engine  data.  ATCOM  used  the 
history  recorder  data,  along  with  data  recorded  in  the  engine  log  book  and  data 
received  from  maintenance  organizations,  to  reconstruct  component  and  module 
historical  records.  ATCOM  also  used  the  data  to  determine  the  causes  of  engine 
removals  through  maintenance  analysis,  to  determine  age  distribution,  and  to 
control  the  maximum  allowable  operating  time  between  overhaul  of  the  Apache 
and  Blackhawk  helicopters.  Additionally,  ATCOM  used  the  historical  analysis 
stored  data  and  determined  that  field  maintenance  organizations  removed 
67  engines  from  helicopters  because  of  low  engine  pressures  and  removed 
41  engines  because  of  foreign  object  damage.  In  addition  to  reporting  the 
collected  data  to  ATCOM,  maintenance  personnel  at  intermediate  and 
organizational  level  maintenance  organizations  furnished  the  data  to  General 
Electric  to  determine  useful  life  of  engine  components. 

As  of  August  1995,  about  1,956  Apache  and  Blackhawk  helicopters  were  in  the 
Army  inventory.  A  total  of  213  field  organizations  were  required  to  collect 
history  recorder  data  from  the  helicopters  and  submit  the  data  to  ATCOM 
monthly.  Our  examination  of  monthly  reports  showed  that  although  213  field 
organizations  were  required  to  report  history  recorder  data  to  ATCOM,  only 
50  regularly  reported  to  ATCOM.  The  remaining  163  field  organizations 
reported  1  to  7  times  during  January  through  August  1995.  For  example, 
32  field  organizations  reported  history  recorder  data  to  ATCOM  once  in 
8  months. 

We  attributed  the  inconsistent  reporting  of  data  to  the  lack  of  ATCOM 
oversight.  ATCOM  personnel  did  not  monitor  the  collection  of  the  history 
recorder  data,  stating  that  data  collection  was  not  monitored  because  it  was  a 
low  priority.  Field  maintenance  personnel  did  not  submit  data  because  they 
believed  the  data  were  not  used,  and  ATCOM  had  not  provided  them  feedback 
concerning  the  data.  As  a  result,  for  engines  in  which  history  recorder  data 
were  not  reported,  ATCOM  lacked  the  necessary  data  to  reconstruct  component 
or  module  historical  records.  ATCOM  was  unable  to  present  the  true  picture  of 
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the  causes  of  engine  removals  and  the  age  of  helicopter  engine  components  with 
limited  life  because  only  24  percent  of  the  Army  field  organizations  were 
reporting  history  data  to  ATCOM  regularly.  The  Army  spent 
approximately  $91,000  annually  for  collecting,  processing,  and  reporting 
history  recorder  data.  Despite  all  that  the  Army  spent,  ATCOM  had  no  up-to- 
date  information  on  whether  helicopters  were  flying  over  the  maximum 
operating  hours  limit  and  thereby  exposing  the  crew  and  aircraft  to  unnecessary 
risks. 


Navy.  The  Naval  Air  Systems  Command  is  responsible  for  the 
management  and  oversight  of  the  Navy's  EMS  program.  Naval  Air  Systems 
Command  Instruction  4790.21  states  that  the  NALDA  system  will  be  used  to 
report  and  store  historical  data.  The  NALDA  system  is  the  Navy's  major 
logistics  information  system  used  by  the  Naval  Air  Systems  Command.  It 
provides  remote  hardware,  telecommunications,  central  computing,  and  a 
database  management  system  for  fleet  support  operations.  The  NALDA  system 
receives  configuration,  maintenance,  material,  operations,  readiness,  safety, 
supply,  and  other  logistics  data  from  existing  data  collection  systems.  It  places 
the  data  in  a  central,  integrated  data  bank  especially  developed  and  structured 
for  the  Naval  Air  Systems  Command  application  programs.  The  NALDA 
system  reformats  EMS  data  received  from  the  Navy  intermediate  level 
maintenance  before  transmitting  the  data  to  the  central  data  base.  However,  the 
Naval  Air  Systems  Command  provides  little  oversight  over  the  EMS  program. 

The  EC  AMS,  a  ground  based  computer,  transmitted  EMS  data  to  the  NALDA 
system.  ECAMS  provided  a  direct  communications  link  between  organizational 
and  intermediate  level  maintenance  organizations.  As  of  September  1995, 
18  intermediate  level  maintenance  organizations  were  supporting  the  Navy 
F/A-18  aircraft.  Of  the  18  organizations,  7  (39  percent)  were  not  reporting  their 
data  daily  to  the  central  data  base.  The  seven  organizations  were  from  15  days 
to  207  days  late  in  reporting  their  data.  Personnel  at  the  central  collection  point 
indicated  that  late  reporting  of  ECAMS  data  adversely  affected  logistics 
management.  Late  submissions  caused  information  on  monitoring  of  engines 
components  with  life  limits  to  be  outdated,  obscured  unit  ownership  of  engines, 
and  reduced  the  quality  and  quantity  of  engineering  data  available  for  analysis  at 
specific  intervals.  Further,  die  fleet  commanders'  efforts  to  determine  the  state 
of  aircraft  readiness  were  hindered  because  outdated  data  did  not  disclose  actual 
engine  conditions  and  events.  For  example,  at  Naval  Air  Station  Atsugi,  Japan, 
the  intermediate  level  maintenance  units  did  not  report  EMS  data  from 
February  27  to  September  21,  1995.  The  Naval  Air  Station  was  207  days  late 
in  reporting  engine  data.  As  a  result  of  not  reporting  engine  data,  the  Navy  did 
not  have  sufficient  logistics  visibility  over  47  engines,  valued  at  $97  million, 
and  19  aircraft,  valued  at  $817  million.  Personnel  at  the  Naval  Air  Station 
stated  that  squadron  ECAMS  operators  did  not  have  time  to  enter  transactions  in 
the  ground  computer  systems  because  the  squadron's  main  priority  was 
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preparing  the  aircraft  to  accomplish  its  assigned  mission.  The  Navy  spent 
approximately  $8.2  million  annually  for  collection,  processing,  and  reporting  of 
EMS  data. 

Air  Force.  The  propulsion  product  group  manager  at  the  San  Antonio 
Air  Logistics  Center  is  responsible  for  the  management  and  oversight  of  the 
Air  Force  EMS  program.  Air  Force  Instruction  21-104  assigns  EMS 
management  responsibility;  however,  Technical  Order  00-25-254-2, 
"Comprehensive  Engine  Management  System,"  February  15,  1993,  states  that 
the  Air  Force  Materiel  Command  is  the  functional  manager  of  the 
comprehensive  engine  management  system,  which  is  the  Air  Force  central  data 
bank  for  engine  management.  The  Oklahoma  City  Air  Logistics  Center  is  the 
system  manager  of  the  central  data  bank. 

The  CEMS  IV,  the  Air  Force  standard  ground  system  for  EMS,  is  a  part  of  the 
comprehensive  engine  management  system.  CEMS  IV  extracts  parts  life 
tracking  data  from  the  inflight  EMS  data  during  engine  download  processing. 
The  data  include  engine  operating  times  and  cycle  counts,  and  allow  the 
calculation  of  the  time  remaining  on  major  assemblies  and  subassemblies.  The 
base-level  engine  management  branches  reported  81  percent  of  the  data  directly 
on-line  by  computer  and  batch  reported  the  remaining  19  percent  on  a  daily 
basis  to  die  central  data  bank.  We  reviewed  the  parts  life  tracking  reports  for 
34  engines  at  Dyess  Air  Force  Base  and  10  engines  at  Shaw  Air  Force  Base. 
Each  of  the  bases  reported  directly  on-line  to  the  central  data  bank,  and  none  of 
the  engine  components  had  reached  the  maximum  life  limits.  Therefore,  the 
reporting  and  reconciling  of  parts  life  tracking  data  from  the  bases  to  the  central 
data  bank  was  adequate.  The  central  data  bank's  cost  for  configuration 
management  and  parts  life  tracking  in  FY  1995  was  $632,000.  The  Air  Force 
did  not  track  the  cost  to  collect  and  process  CEMS  IV  data  on  an 
Air  Force-wide  basis;  however,  the  cost  for  the  combined  CEMS  IV  data 
collection  and  processing  costs  for  Dyess,  Seymour  Johnson,  and  Shaw 
Air  Force  Bases  was  about  $175,000  for  1995.  Overall,  the  Air  Force 
centralized  collecting  and  reporting  of  EMS  data  were  well  managed. 

EMS  Performance  Measures.  The  Military  Departments  had  not  developed 
useful  performance  measures  for  determining  the  cost  and  operational 
effectiveness  of  EMS.  The  Military  Departments  had  no  criteria  or  guidance 
for  determining  whether  the  EMS  met  program  needs  or  were  cost-effective. 
For  example,  the  Military  Departments  did  not  have  measurement  criteria  to 
determine  whether  the  EMS  reduced  unscheduled  engine  removals,  maintenance 
manhours,  or  air  and  ground  aborts.  Additionally,  they  did  not  have  tracking 
measures  to  determine  the  success  or  failure  of  maintenance  actions  based  on 
EMS  generated  data.  As  a  result,  the  Military  Departments  had  no  means  of 
determining  the  effectiveness  of  their  EMS  programs. 
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EMS  Technical  Data  and  Training 


EMS  technical  data  updating  procedures  in  the  Air  Force  and  EMS  training  in 
the  three  Military  Departments  needed  improvement.  The  condition  existed 
because  of  inadequate  guidance  and  management  oversight. 

Updating  EMS  Technical  Data.  The  Army  and  the  Navy  provided  current  and 
complete  technical  data  to  organizational  level  maintenance  personnel,  but  the 
Air  Force  can  improve  in  this  regard.  The  technical  orders  for  the  EMS  on  the 
Army  Apache  and  Blackhawk  helicopters  and  the  Navy  F-14,  F/A-18,  and  the 
Air  Force  F-16  aircraft  engines  were  current  and  complete.  However,  the  Air 
Force  did  not  ensure  that  the  technical  orders  for  the  EMS  on  the  F-15E  and 
B-1B  aircraft  engine  provided  current  and  complete  procedures  for  fault 
isolation. 

The  purpose  of  EMS  technical  orders  is  to  provide  technical  information  and 
directions  required  to  perform  fault  isolation  procedures  on  engine  malfunctions 
(faults)  that  EMSs  detect.  Each  fault  isolation  procedure  for  an  automated  EMS 
begins  with  a  recorded  fault  message  and  terminates  with  corrective  action 
instructions.  Troubleshooting  procedures  contained  in  the  technical  orders  are 
based  on  the  analysis  of  EMS  and  engine  malfunctions  and  reflect  the 
knowledge  gained  from  field  experience.  Another  purpose  of  the  technical 
orders  is  to  guide  the  user  in  trend  data  analysis,  if  applicable,  for  an  engine 
type.  Technical  orders  for  EMS  were  designed  to  lead  the  user  through  the 
trend  analysis  process  from  data  collection  to  troubleshooting. 

The  Air  Force  did  not  ensure  the  timely  incorporation  of  technical  order 
changes  resulting  from  software  logic  upgrade  2.3. OA,  "Intermittent  Logic," 
for  the  F-15E  engine  diagnostic  unit.  The  upgrade  was  installed  in  May  1993. 
However,  the  contractor  did  not  submit  a  request  to  change  the  applicable 
technical  order  until  January  1995.  The  Air  Force  did  not  provide  maintenance 
personnel  at  Seymour  Johnson  Air  Force  Base  with  the  changes  in  fault  isolation 
and  troubleshooting  procedures  until  August  1995.  The  delay  occurred  because 
the  Air  Force  did  not  ensure  that  the  contractor  issued  the  requests  to  change  the 
technical  orders  on  time.  As  a  result,  maintained  had  nothing  to  direct  them  to 
delay  noncritical  repairs  (deficiencies  not  critical  enough  to  ground  the  aircraft) 
until  the  end  of  the  day. 

The  Air  Force  did  not  publish  a  technical  order  for  the  EMS  on  the  B-1B 
aircraft  until  July  1995,  even  though  the  EMS  was  fielded  in  1986.  Air  Force 
personnel  have  been  responsible  for  B-1B  maintenance  at  the  organizational 
level  since  the  aircraft  was  fielded  and  have  collected  the  performance  trend 
data  needed  to  develop  the  EMS  technical  orders.  However,  the  data  served 
primarily  as  a  tool  for  the  contractor  to  develop  a  trend  monitoring  program 
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rather  than  to  develop  Air  Force  EMS  technical  orders.  As  a  result,  from  1986 
to  1995,  the  Air  Force  did  not  have  EMS  technical  orders  for  the  B-1B  engines 
and  could  not  optimize  the  use  of  the  B-1B  trending  data  to  make  maintenance 
decisions. 

Training  in  the  Use  of  Ground  Processing  Systems.  The  Military 
Departments  did  not  emphasize  training  on  EMS  issues.  The  Military 
Departments  needed  additional  training  to  collect  and  report  accurate  EMS  data, 
use  ground-based  computer  systems  and  download  devices,  and  use  EMS 
trending  data,  where  available,  to  monitor  for  performance  degradation  and  the 
associated  engine  degradation. 

Army.  Maintenance  personnel  within  the  Army  received  some  training 
on  EMS  issues.  However,  Army  maintenance  personnel  needed  refresher 
training  on  the  importance  of  reporting  timely  data  to  the  central  data  base.  For 
example,  the  Army  used  the  history  recorder  for  tracking  life-limited 
components,  a  process  requiring  accurate  and  timely  reporting.  However,  only 
50  of  213  maintenance  organizations  that  were  required  to  submit  reports  to  the 
central  data  base  regularly  submitted  the  data. 

Navy.  Although  the  Navy  maintenance  personnel  received  formal 
training  on  the  operation  of  the  ECAMS  ground-based  computer,  they  needed 
additional  training  on  reporting  and  using  data  generated  by  ECAMS.  For 
example,  on  September  21,  1995,  Naval  Air  Station  Rota,  Spain,  maintenance 
personnel  transmitted  five  transactions  involving  engine  number  E3 10900  to  the 
central  collection  point.  The  central  collection  point  did  not  accept  the  five 
transactions  because  maintenance  personnel  at  the  Naval  Air  Station  Rota, 
Spain,  did  not  correct  prior  errors  on  the  engines,  such  as  incorrect  component 
number,  incorrect  serial  number,  and  incorrect  asset  locations.  The  original 
transactions  with  errors  were  transmitted  on  July  24,  1995.  Because  the  errors 
were  not  corrected,  all  transactions  transmitted  after  that  date  for  engine  number 
E3 10900  were  not  processed.  Because  the  central  data  base  was  updated  daily, 
data  concerning  the  specific  engine  was  not  complete,  and  the  status  of  the 
engine  remained  in  doubt.  We  also  analyzed  the  frequency  of  use  of  ECAMS 
reports  for  the  period  May  through  September  1995.  Of  the  12  different  types 
of  reports  that  could  be  generated  from  the  ECAMS  ground  computer,  the  Navy 
used  only  3  of  the  reports  on  a  consistent  basis.  It  used  the  reports  to  obtain 
engine  hardware  configuration  without  having  to  visually  inspect  the  engine,  to 
estimate  spare  parts,  and  to  locate  alternate  sources  of  spare  parts.  However, 
the  Navy  seldom  used  the  report  that  would  resolve  data  entries  flagged  as 
errors  in  the  data  base.  Central  collection  point  personnel  indicated  that  the  low 
use  of  the  error  correction  report  was  caused  by  a  lack  of  adequate  training  in 
understanding  and  using  various  reports  available  through  the  ECAMS  ground 
computer. 
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Air  Force.  Air  Force  maintenance  personnel  had  limited  training  on 
EMS  data  collection  and  analysis  procedures.  The  Air  Force  used  tactical 
aircraft  maintenance  (TAM)  personnel  to  perform  aircraft  engine  maintenance 
and  EMS  duties.  Air  Force  Manual  36-2108,  "Airman  Classification  Manual," 
October  31,  1994,  states  that  the  job  responsibilities  of  TAM  personnel  include 
maintaining  tactical  aircraft,  support  equipment,  and  forms  and  records.  The 
manual  also  states  that  the  TAM  personnel  must  use  technical  data  to  diagnose 
and  solve  maintenance  problems  on  the  majority  of  systems  within  the  aircraft, 
including  engines.  However,  TAM  personnel  did  not  receive  extensive  basic 
engine  training  to  become  proficient  in  engine  maintenance  or  in  EMS  duties. 
Additionally,  the  overall  aircraft  systems  orientation  of  the  Air  Force  specialty 
code  did  not  permit  the  depth  of  engine  work  experience  necessary  to  establish 
proficiency  in  those  two  areas.  Further,  TAM  personnel  assigned  as  EMS 
monitors  generally  did  not  have  specialized  training  in  the  use  of  the  CEMS  IV, 
the  Air  Force  standard  ground  system  for  EMS. 

Basic  Engine  Training.  The  TAM  personnel  who  operated  the 
Air  Force  EMS  equipment  were  not  adequately  trained  to  interpret  and  properly 
use  diagnostic  data  from  the  EMS  download  devices  or  to  use  CEMS-IV. 
However,  they  had  to  use  the  fault  codes  produced  by  the  EMS  hardware  to 
identify  the  appropriate  maintenance  procedures  and  to  troubleshoot  engine 
problems.  For  instance,  at  Seymour  Johnson  and  Shaw  Air  Force  Bases,  only 
266  (35  percent)  of  the  757  TAM  personnel  assigned  had  attended  courses  for 
EMS  familiarization,  fault  isolation,  or  maintenance  procedures  for  the  specific 
engines  that  they  repaired.  Additionally,  of  the  one  bomber  and  two  fighter 
wings  that  we  visited,  only  one  of  the  20  TAM  technicians  acting  as  EMS 
monitors  had  attended  the  formal  CEMS  IV  course. 

Neither  the  TAM  personnel  at  the  organizational  level  who  used  diagnostic  data 
to  troubleshoot  engine  discrepancies  nor  the  EMS  monitors  who  operated 
CEMS  IV  received  adequate  training.  The  units  allowed  the  maintenance 
personnel  to  attend  training  as  the  unit  work  load  permitted.  However, 
workload  demands  and  a  workforce  composed  of  237  (31  percent)  of 
757  entry-level  airmen  resulted  in  training  receiving  less  emphasis.  Inadequate 
engine-specific  training  was  not  an  issue  at  Dyess  Air  Force  Base,  which  used 
engine  specialists  at  the  organizational  level.  Dyess  Air  Force  Base  was  also  the 
only  installation  of  the  three  reviewed  that  had  an  EMS  monitor  who  had 
attended  the  formal  CEMS  IV  course. 

Engine  Maintenance.  During  May  1995,  the  fielded  engine 
supportability  integrated  product  team  chartered  by  the  designated  acquisition 
commander  at  the  San  Antonio  Air  Logistics  Center  began  assessing  the  use  of 
TAM  personnel  to  perform  engine  maintenance  at  the  organizational  level.  The 
consensus  of  the  team  was  that  the  TAM  concept  did  not  work  for  engine 
maintenance;  therefore,  trained  propulsion  technicians  should  support  the 
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flightline.  As  a  result,  the  program  manager  for  engine  trending  and  diagnostics 
contracted  with  the  Science  Applications  International  Corporation  to  determine 
the  best  way  to  train  both  the  existing  TAM  workforce  and  engine  specialists  to 
achieve  full  proficiency.  The  study  is  being  conducted,  with  an  estimated 
completion  date  of  August  1,  1996. 


Summary 


The  Military  Departments  needed  to  improve  their  EMS  programs.  EMS  used 
within  the  Military  Departments  did  not  provide  the  information  needed  to  fully 
evaluate  the  condition  of  the  engines.  For  an  EMS  to  be  effective,  it  must 
provide  information  needed  to  diagnose,  monitor,  and  trend  the  condition  of 
engines.  The  requirements  should  be  addressed  in  the  guidance  and 
performance  specifications  used  for  developing  the  system.  The  EMSs  used  on 
engines  of  the  Army  Apache  and  Blackhawk  helicopters  was  limited  and  did  not 
provide  critical  diagnostic  and  trending  data.  Also,  the  Navy  EMS  ground 
system  for  the  F-14  and  F/A-18  aircraft  had  diagnostic  and  parts  life  tracking 
capabilities,  but  no  trending  capabilities,  and  the  EMSs  on  the  Air  Force  B-1B, 
F-15E,  and  F-16C/D  did  not  have  full  trending  capabilities.  However,  with 
effective  oversight  by  the  Deputy  Under  Secretary  of  Defense  for  Logistics  and 
the  Military  Departments,  EMS  under  development  will  correct  some  of  the 
EMS  deficiencies.  The  Army  and  Navy  need  to  ensure  consistent  and  timely 
collection  of  EMS  data,  and  the  Military  Departments  need  to  determine  the 
cost  and  operational  effectiveness  of  the  EMS  programs.  Finally,  the  Air  Force 
needs  to  ensure  timely  and  complete  updating  of  technical  data,  and  all  Military 
Departments  need  to  improve  EMS  training  programs. 


Recommendations  for  Corrective  Action 


1.  We  recommend  that  the  Deputy  Under  Secretary  of  Defense  for 
Logistics  provide  management  oversight  of  the  Joint  Advanced  Health  and 
Usage  Monitoring  System  program  to  ensure  that  the  Army  and  the  Navy 
collect,  process,  record,  and  effectively  use  data  generated  from  the  system 
to  detect  and  predict  engine,  component,  and  structural  failures  before  they 
occur. 

Deputy  Under  Secretary  of  Defense  for  Logistics  Comments.  The  Deputy 
Under  Secretary  of  Defense  for  Logistics  concurred  with  the  recommendation 
and  stated  that,  in  the  case  of  the  Joint  Advanced  Health  and  Usage  Monitoring 


16 


Management  of  Engine  Monitoring  Systems  Programs 


System,  the  Deputy  Under  Secretary  of  Defense  for  Logistics  will  provide 
management  oversight  in  coordination  with  the  Deputy  Under  Secretary  of 
Defense  (Advanced  Technology).  The  Deputy  Under  Secretary  of  Defense  for 
Logistics  also  stated  that  the  Joint  Advanced  Health  and  Usage  Monitoring 
System  is  scheduled  for  initiation  in  FY  1997.  Specific  plans  are  dependent 
upon  FY  1997  funding. 

2.  We  recommend  that  the  Commander,  Aviation  and  Troop  Command; 
Commander,  Naval  Air  Systems  Command;  and  the  Air  Force  Propulsion 
Product  Group  Manager  issue  guidance  requiring  that  performance 
specifications  be  developed  for  all  engine  monitoring  systems  under 
development.  The  specifications  should  ensure  that  the  systems  generate 
the  data  needed  to  monitor,  diagnose,  and  trend  the  condition  of  engines. 

Army  Comments.  The  Army  concurred  with  the  recommendation,  stating  that 
the  current  EMS  program  for  the  T800  engine  is  being  developed  to 
performance  specifications  that  will  ensure  that  the  appropriate  data  are 
generated  to  monitor  the  engine  condition,  diagnose  fault,  and  trend  engine 
performance. 

Audit  Response.  We  consider  the  Army's  comments  regarding  the 
development  of  performance  specifications  for  the  EMS  program  currently 
under  development  to  be  positive  a  step.  However,  the  Army's  comments  were 
not  fully  responsive  because  the  comments  did  not  address  the  need  to  issue 
guidance  requiring  performance  specifications  for  current  and  future  EMS 
developments  and  did  not  indicate  a  completion  date  for  actions  being  taken. 
The  intent  of  our  recommendation  was  to  ensure  that  future  engine  monitoring 
systems  are  not  developed  without  performance  specifications.  We  request  that 
the  Army  provide  additional  comments  and  a  completion  data  for  actions  to  be 
taken  in  its  response  to  the  final  report. 

Navy  and  Air  Force  Comments.  The  Navy  and  the  Air  Force  did  not  respond 
to  the  draft  report.  We  request  that  the  Navy  and  the  Air  Force  provide 
comments  in  response  to  the  final  report. 

3.  We  recommend  that  the  Commander,  Aviation  and  Troop  Command, 
and  the  Commander,  Naval  Air  Systems  Command,  selectively  test  the 
consistency  and  timeliness  of  engine  monitoring  system  data  reported  by 
maintenance  organizations.  Where  determined  necessary,  additional 
training  should  be  provided  to  maintenance  personnel  on  timely  reporting 
of  engine  monitoring  system  data. 

Army  Comments.  The  Army  concurred,  stating  that  action  had  been  initiated 
to  improve  EMS  data  reporting  for  the  T-700  engine.  The  Army  also  stated  that 
requirements  had  been  submitted  to  the  program  manager  for  development  and 
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fielding  of  the  Unit  Level  Logistics  System-Aviation  for  automated  electronics 
transmission  of  EMS  data.  The  system  is  currently  being  fielded  and  electronic 
transmission  of  data  should  begin  in  FY  1998. 

4.  We  recommend  that  the  Commander,  Aviation  and  Troop  Command; 
the  Commander,  Naval  Air  Systems  Command;  and  the  Air  Force 
Propulsion  Product  Group  Manager  develop  guidance  to  establish  useful 
performance  measures  to  determine  the  cost  and  operational  effectiveness 
of  the  engine  monitoring  system  programs  already  in  existence  and  those 
under  development.  Special  emphasis  should  be  placed  on  criteria  for 
determining  whether  the  engine  monitoring  system  reduces  unscheduled 
engine  removals,  maintenance  manhours,  or  air  and  ground  aborts. 

Army  Comments.  The  Army  partially  concurred,  stating  that  with  regard  to 
the  new  system  under  development,  ATCOM  will  perform  an  analysis  of  the 
T800  EMS  data  to  assess  the  value  added  by  the  diagnostic  system.  The 
analysis  will  require  the  engines  to  be  fielded  for  some  period  of  time  to 
develop  failure  diagnostic  history  and  will  be  contingent  upon  funding  support. 
With  regard  to  the  system  in  existence,  the  history  recorder  (counter)  is  not  an 
EMS  and  was  not  designed  to  provide  diagnostics  and  trending  capabilities; 
therefore,  analysis  to  assess  value  added  is  not  feasible. 

Audit  Response.  The  Army's  comments  addressing  the  analysis  that  will  be 
performed  on  the  T800  EMS  data  to  assess  the  value  added  by  the  diagnostic 
system  will  be  quite  beneficial  to  the  Army.  However,  the  Army's  comments 
were  not  fully  responsive  because  the  comments  did  not  address  the  need  to 
develop  guidance  establishing  useful  performance  measures  for  determining  the 
cost  and  operational  effectiveness  of  EMSs  and  did  not  indicate  a  completion 
data  for  actions  being  taken.  The  intent  of  our  recommendation  was  to  ensure 
that  useful  performance  measures  are  put  in  place  for  measuring  the 
effectiveness  of  EMSs.  While  it  is  recognized  that  the  history  recorder 
(counter)  is  not  an  EMS,  there  should  be  performance  measures  for  determining 
its  cost  and  operational  effectiveness  because  the  history  recorder  (counter)  is 
widely  used  on  the  Army's  helicopters.  We  request  that  the  Army  provide 
additional  comments  and  a  completion  date  for  actions  to  be  taken  in  response  to 
the  final  report. 

Navy  and  Air  Force  Comments.  The  Navy  and  the  Air  Force  did  not  respond 
to  the  draft  report.  We  request  that  the  Navy  and  the  Air  Force  provide 
comments  in  response  to  the  final  report. 

5.  We  recommend  that  the  Air  Force  Propulsion  Product  Group  Manager 
initiate  a  review  of  the  engine  monitoring  system  technical  order  change 
process.  The  technical  orders  should  be  reviewed  for  the  timely 
incorporation  of  changes  to  maintenance  procedures. 
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6.  We  recommend  that  the  Commander,  Naval  Air  Systems  Command, 
provide  additional  training  to  maintenance  personnel  on  the  operation  of 
systems  used  to  collect  and  report  engine  monitoring  systems  data.  Special 
emphasis  should  be  placed  on  correcting  errors  in  data  generated  by  the 
systems. 

7.  We  recommend  that  the  Air  Force  Propulsion  Product  Group  Manager: 

a.  Determine  engine  monitoring  system  training  requirements. 

b.  Establish  follow-on  training  for  personnel  performing  engine 
maintenance.  The  training  should  cover  engine-specific  maintenance  and 
engine  monitoring  system  requirements  for  each  engine  maintained. 

c.  Work  with  the  Air  Force  Education  and  Training  Command  to 
provide  the  required  training  to  engine  monitoring  system  users. 

Navy  and  Air  Force  Comments.  The  Navy  and  the  Air  Force  did  not  respond 
to  the  draft  report.  We  request  that  the  Navy  and  the  Air  Force  provide 
comments  in  response  to  the  final  report. 
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Part  II  -  Additional  Information 


Appendix  A.  Audit  Process 


Scope 


We  performed  the  audit  at  the  Office  of  the  Deputy  Under  Secretary  of  Defense 
for  Logistics,  Army,  Navy,  and  Air  Force  organizations  with  responsibilities 
related  to  EMS  programs.  We  reviewed  policies,  procedures,  and  practices  for 
developing  new  EMS  and  for  managing  existing  EMS.  We  evaluated  the 
history  recorder  installed  on  the  Army  Apache  and  Blackhawk  helicopter 
engines.  We  also  evaluated  more  complex  EMS,  which  were  installed  on  the 
Navy  F-14B/D  and  F/A-18A/D  aircraft  engines,  and  on  the  Air  Force  B-1B,  F- 
15E,  and  F-16C/D  aircraft  engines.  Additionally,  we  reviewed  a  proposed 
monitoring  system  that  the  Army  and  the  Navy  will  jointly  develop  for 
installation  on  Army  and  Navy  helicopters.  Further,  we  evaluated  training 
practices  for  personnel  assigned  to  EMS  related  duties.  Army,  Navy,  and  Air 
Force  records  reviewed  covered  calendar  years  1994  and  1995.  We  also 
reviewed  a  briefing  on  the  Joint  Advanced  Health  and  Usage  Monitoring  System 
that  was  presented  to  the  Deputy  Under  Secretary  of  Defense  for  Logistics  on 
March  19,  1996.  In  addition,  we  discussed  the  benefits  of  EMS  and  overall 
management  of  the  EMS  program  with  responsible  officials  at  all  organizational 
levels. 


Methodology 


We  judgmentally  selected  eight  aircraft  engines  and  evaluated  EMS  used  on 
those  engines.  We  also  reviewed  the  EMS  that  are  under  development  in  each 
Military  Department  and  a  proposed  joint  Army  and  Navy  monitoring  system. 
Specifically,  we  evaluated  practices  for  gathering  engine  performance  data, 
transmitting  the  data  to  ground  processing  systems,  and  analyzing  the  data  to 
detect  engine-related  problems  and  identify  required  maintenance.  We  also 
reviewed  judgmentally  selected  samples  of  records  for  maintenance  history, 
engine  health  profiles,  time  changes  and  inspections,  and  manpower  and 
training.  We  did  not  attempt  to  correlate  engine  failure  data  with  the 
management  of  EMS,  because  we  did  not  collect  data  that  suggested  failures 
could  have  been  prevented. 
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Use  of  Computer-Processed  Data.  Except  to  verify  the  accuracy  of 
judgmentally  selected  samples  of  records  for  maintenance  history,  engine  health 
profiles,  time  changes  and  inspections,  and  manpower  and  training,  we  made  no 
independent  assessment  of  the  overall  reliability  of  computer-processed  data 
used  in  managing  EMS.  As  discussed  in  the  report,  there  were  problems  in  the 
Army  and  Navy  with  consistent  and  timely  input  of  data  to  centralized  data 
bases.  Reliability  problems  with  computer-processed  data  would  not  have 
affected  the  results  of  our  audit. 

Technical  Assistance.  Technical  assistance  was  provided  by  the  Analysis, 
Planning  and  Technical  Support  Directorate,  Technical  Assessment  Division. 
Technical  assistance  was  provided  in  evaluating  the  effectiveness  of  EMS  being 
developed  and  systems  being  used  to  monitor  the  condition  of  jet  aircraft 
engines. 

Audit  Period,  Standards,  and  Locations.  We  performed  this  economy  and 
efficiency  audit  from  May  1995  through  March  1996  in  accordance  with 
auditing  standards  issued  by  the  Comptroller  General  of  the  United  States,  as 
implemented  by  the  Inspector  General,  DoD.  Accordingly,  we  included  tests  of 
management  controls  considered  necessary.  Appendix  E  lists  the  organizations 
we  visited  or  contacted. 


Management  Control  Program 


DoD  Directive  5010.38,  "Internal  Management  Control  Program,"  April  14, 
1987,  requires  DoD  organizations  to  implement  a  comprehensive  system  of 
management  controls  that  provides  reasonable  assurance  that  programs  are 
operating  as  intended  and  to  evaluate  the  adequacy  of  the  controls. 

Scope  of  Review  of  Management  Controls.  The  audit  evaluated  management 
controls  related  to  the  development  and  management  of  EMS  within  the  Military 
Departments.  Specifically,  we  examined  the  management  control  procedures 
for  ensuring  that  maintenance  personnel  were  provided  with  the  necessary  data 
and  adequately  trained  on  the  systems  to  perform  maintenance  on  jet  aircraft 
engines.  We  did  not  assess  the  adequacy  of  management’s  self-evaluation  of 
those  controls  because  management  did  not  identify  its  EMS  programs  under 
one  or  more  of  its  assessable  units. 

Adequacy  of  Management  Controls.  We  identified  material  management 
control  weaknesses  as  defined  by  DoD  Directive  5010.38.  The  Military 
Departments  had  not  established  effective  management  controls  to  ensure  that 
existing  EMS  programs  provided  maintenance  personnel  with  the  necessary  data 
and  adequate  training  on  the  systems  for  performing  maintenance  on  jet 
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aircraft  engines.  All  recommendations,  if  implemented,  will  correct  the 
material  weaknesses.  A  copy  of  this  report  will  be  provided  to  the  senior 
officials  in  charge  of  management  controls  in  the  Office  of  the  Secretary  of 
Defense  and  the  Military  Departments. 

Adequacy  of  Management's  Self-Evaluation.  The  Military  Departments  did 
not  identify  development  and  management  of  EMS  as  assessable  units,  and 
therefore,  did  not  identify  or  report  the  material  management  control 
weaknesses  identified  by  the  audit. 
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The  Inspector  General,  DoD,  issued  Report  No.  94-133,  "Obtaining  the 
Maximum  Life  From  F-404  Jet  Engine  Components,"  on  June  14,  1994.  The 
report  stated  that  the  Navy  replaced  F-404  life-limited  engine  components  even 
though  a  high  probability  (99.9  percent)  existed  that  the  components  had 
additional  life  remaining.  We  estimated  that  by  using  an  inspection  program  to 
manage  the  engines,  the  Navy  could  avoid  the  procurement  of  $75.5  million  of 
replacement  components  and  achieve  a  net  savings  of  $52.4  million  over  the 
remaining  life  cycle  of  the  F/A-18  aircraft  (15  years).  The  report  recommended 
that  the  Navy  establish  a  program  of  periodic  inspections  of  F-404  engine 
components  to  optimize  engine  component  life  and  to  ensure  efficient  use  of 
resources.  The  Navy  nonconcurred  with  the  recommendation.  Although  the 
Navy  will  not  implement  an  inspection  program  for  F-404  engines,  in  the  future 
the  Navy  will  evaluate  the  use  of  an  inspection  program,  such  as  the  retirement 
for  cause  program,  for  newly  developed  and  redesigned  engines  (including  the 
F-404  engine).  Whenever  feasible  and  advantageous,  the  Navy  will  incorporate 
an  inspection  program  into  the  maintenance  concept  for  engines. 

The  Inspector  General,  DoD,  Report  No.  94-045,  "Life  Reduction  of 
T700  Aircraft  Engine  Components,"  was  issued  on  February  25,  1994.  The 
report  stated  that  Army  and  Marine  Corps  operational  units  were  flying 
78  helicopters  with  T700  components  that  had  exceeded  manufacturer's  revised 
recommended  interim  life  limits.  If  the  interim  limits  are  proved  to  be  correct, 
possible  failure  of  the  engine  components  could  result  in  damage  to  the  aircraft 
and  loss  of  life.  The  report  recommended  that  the  Army  and  Navy  issue 
appropriate  safety  of  flight  messages  to  field  units  informing  field  maintenance 
personnel  of  the  new  interim  life  limits  that  General  Electric  is  recommending 
for  the  T700  engine  and  require  maintenance  units  to  remove  from  aircraft 
engines  those  components  that  have  reached  the  new  interim  life  limits 
recommended  by  General  Electric. 

The  Army  partially  concurred  with  the  recommendations  and  planned  to  publish 
a  revised  technical  bulletin  and  to  remove  from  its  engines  any  T700 
components  that  have  exceeded  the  limits  published  in  the  bulletin.  Technical 
Bulletin  1-2840-248-20-18  was  issued  on  May  31,  1994.  The  technical  bulletin 
is  sufficient  to  accomplish  the  intended  outcome.  It  requires  all  T700-GE-700 
and  T700-GE-701  engine  components  that  exceeded  the  applicable  life  limits  to 
be  removed. 

The  Navy  nonconcurred  with  the  recommendations,  but  stated  it  would  leave 
current  life  limits  in  effect  while  General  Electric  and  the  Navy  review  the 
assumptions  of  the  methodology.  A  dialogue  with  Navy  officials  to  resolve  that 
matter  was  ongoing  as  of  March  1996. 
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The  Inspector  General,  DoD,  Report  No.  94-041,  "Warranties  for  the 
Navy  F-404  Jet  Aircraft  Engine,"  was  issued  on  February  14,  1994.  The 
report  stated  that  although  the  Navy  invoked  the  warranty  provisions  to  obtain 
reimbursement  for  the  life  it  will  not  achieve  from  nine  defective  F-404  engine 
components,  it  had  not  invoked  the  warranty  provisions  to  obtain  compensation 
(including  redesign  costs)  for  other  defective  components  that  are  covered  by 
warranty.  As  a  result,  the  Navy  can  seek  an  estimated  $10.6  million  of 
additional  compensation  from  General  Electric  for  replacement  and  redesign  of 
engine  components.  The  report  recommended  that  the  Navy  invoke  the 
provisions  of  the  warranty  that  require  General  Electric  to  redesign  or  replace 
all  F-404  engine  components  that  were  defective.  The  Navy  agreed  to  take  the 
recommended  actions. 
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Engine  Monitoring  System 


The  EMS  isolates  engine  control  system  and  EMS  detected  faults  during  engine 
operation  to  aid  maintenance  personnel  during  engine  troubleshooting.  It 
generates  significant  maintenance  reports  that  can  be  used  for  making  timely 
decisions  about  whether  aircraft  should  fly.  The  EMS  helps  to  reduce  engine 
maintenance  manhours  and  to  increase  aircraft  availability.  The  EMS  consists 
of  the  in-flight  EMS  and  ground  support  equipment  EMS. 

Army 

Apache  and  Blackhawk  Helicopter  EMS.  The  history  recorder  is  a  very 
valuable  and  critical  instrument  for  the  safe  operation  of  the  T-700  helicopter 
engine  series  used  on  the  Army  Apache  and  Blackhawk  helicopters.  It  is  used 
in  tracking  components  with  contractor  established  limited  life.  The  recorder 
provides  data  for  maintenance  personnel  to  compute  the  time  remaining  on  those 
components  before  being  replaced. 

Navy 

F-14  and  F/A-18  Aircraft  EMS.  The  in-flight  EMS  for  the  F-14  and 
F/A-18  records  engine  status  and  tracks  engine  life  cycle  usage,  and  records 
pre-event  and  post-event  engine  data.  An  engine  event  is  an  occurrence  of  any 
of  the  following  conditions:  abnormal  inlet  temperature,  abnormal  oil  pressure, 
flameout,  or  overspeed.  The  in-flight  EMS  is  located  in  the  cockpit  and  alerts 
the  pilot  during  flight  to  serious  engine  anomalies.  It  also  sets  maintenance 
codes  for  the  ground  crew.  The  EMS  provides  ground  crewmen  with  a  3-digit 
number  called  maintenance  monitor  panel  codes  for  each  event  detected  by  the 
EMS  during  the  flight.  1 

Enhanced  Comprehensive  Asset  Management  System.  The  ECAMS  is  a 
ground  computer  system  that  was  developed  to  support  on-condition 
maintenance  for  the  Navy  F/A-18  aircraft  and  its  engine.  The  on-condition 
maintenance  concept  is  based  on  the  Parts  Life  Tracking  System  that  specifies 
items  that  are  tracked  by  their  useful  life  rather  than  a  fixed  number  of  hours. 
The  outcome  is  higher  aircraft  availability  and  lower  maintenance  costs. 
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Maintenance  Data  Processing  System.  The  Navy  is  developing  the 
maintenance  data  processing  system  to  replace  the  preproduction  maintenance 
data  processing  system  and  ECAMS.  The  system  will  greatly  enhance  the 
tracking,  trending,  and  diagnostic  capabilities  of  the  in-flight  EMS.  The 
in-flight  EMS  for  die  F-14B/D  aircraft  uses  the  preproduction  maintenance  data 
processing  system  for  data  processing  and  analysis. 

Air  Force 

B-1B  Aircraft  EMS.  The  central  integrated  test  system  is  the  EMS  for  the 
B-1B  engine.  It  was  fielded  on  the  first  B-1B  aircraft  introduced  into 
operational  service  and  currently  operates  on  all  B-1B  aircraft.  The  Rockwell 
manufactured  system  monitors  a  number  of  aircraft  subsystems.  The 
components  of  the  B- IB  aircraft  EMS  include  the  engine  mounted  processor,  an 
airframe  mounted  computer,  a  maintenance  recorder,  an  airborne  printer,  a  data 
acquisition  unit,  and  CEMS  IV.  The  system  records  several  EMS  data  types, 
including  manually  entered  aircraft  and  engine  identification,  failure  messages, 
failure  data  snapshots,  and  engine  trend  data.  Capabilities  of  the  system  include 
providing  in-flight  data  to  the  B-1B  crew  members  as  well  as  storing  engine 
fault  and  trend  data  on  magnetic  tape  recorder  cartridges. 

F-15E  Aircraft  EMS.  The  F-15E  aircraft  EMS  is  composed  of  two  main 
in-flight  subsystems,  the  digital  electronic  engine  control  and  the  engine 
diagnostic  unit,  and  two  main  ground  subsystems,  a  download  device  called  the 
comprehensive  engine  diagnostic  system  and  CEMS  IV.  The  digital  electronic 
engine  control  continuously  evaluates  the  health  of  the  engine  by  controlling  and 
monitoring  engine  operation.  The  data  flow  from  the  digital  electronic  engine 
control  to  the  engine  diagnostic  unit  alerts  the  system  when  (and  tells  where)  an 
anomaly  has  occurred.  In  addition  to  detecting  failures,  the  digital  electronic 
control  unit  provides  the  engine  diagnostic  unit  with  engine  raw  data  such  as 
pressures,  speed  signals,  and  temperatures.  The  engine  diagnostic  unit 
continuously  reviews  the  digital  electronic  engine  control  output  and  stores  data 
useful  to  die  maintenance  crew,  such  as  engine  time  and  cycle  data,  engine 
performance  parameters,  fault  codes,  and  transient  data.  It  also  stores  and 
records  engine  problem  data  and  assigns  a  time  stamp  (number  of  minutes  after 
engine  start)  that  the  fault  or  event  occurred.  Additionally,  the  engine 
diagnostic  unit  monitors  operating  parameters  that  are  not  directly  measurable 
by  the  digital  electronic  engine  control,  such  as  oil  pressure. 

F-16C/D  Aircraft  EMS.  The  F-16C/D  aircraft  EMS  is  composed  of  an  engine 
mounted  digital  electronic  control;  the  comprehensive  engine  diagnostic  system, 
which  is  a  download  device;  and  the  CEMS  IV  ground  computer  system.  The 
EMS  collects  and  transfers  parts  life  tracking  information,  performance  trending 
information  and  fault  detection  data  to  the  organizational  level  maintenance 
units  for  immediate  use  and  subsequent  transfer  to  the  base  level  engine 
management  branch. 
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Comprehensive  Engine  Management  System  IV.  The  CEMS  IV  is  an  Air 
Force  managed  computer  system  used  to  support  engine  trending  and 
diagnostics  at  the  base  level.  The  system  was  designed  to  improve  maintenance 
efficiency  through  consolidation  of  engine  diagnostic  resources  into  a  single 
useful  data  product  presented  in  a  convenient  format  for  maintenance  personnel. 
It  is  used  for  fault  or  event  detection,  performance  trending,  and  multi-source 
data  correlation.  It  accomplishes  fault  and  event  detection  and  trending  by 
comparing  input  parameters  to  limits  and  by  providing  plotting  capabilities  to 
facilitate  human  interpretation  of  adverse  trends.  CEMS  IV  collects,  stores, 
analyzes,  and  presents  data  from  the  on-board  EMS  and  maintenance  data 
collection  systems  to  distribute  a  comprehensive  view  of  engine  information  to 
all  users  located  at  the  base  level. 


29 


Appendix  D.  Organizations  Visited  or  Contacted 

Office  of  the  Secretary  of  Defense 

Office  of  the  Assistant  Deputy  Under  Secretary  of  Defense  (Maintenance  Policy,  Plans, 
and  Resources),  Washington,  DC 


Department  of  the  Army 

Headquarters,  Deputy  Chief  of  Staff  (Logistics),  Washington,  DC 

Headquarters,  Army  Materiel  Command,  Alexandria,  VA 

Aviation  and  Troop  Command,  St.  Louis,  MO 

18th  Airborne  Corps,  Fort  Bragg,  NC 

Corpus  Christi  Army  Depot,  Coipus  Christi,  TX 

Aviation  Applied  Technology  Directorate,  Fort  Eustis,  VA 


Department  of  the  Navy 

Office  of  the  Deputy  Chief  of  Naval  Operations  (Logistics),  Washington,  DC 
Naval  Air  Systems  Command,  Arlington,  VA 
Naval  Air  Station  Cecil  Field,  Jacksonville,  FL 
Naval  Air  Station  Lemoore,  Lemoore,  CA 
Naval  Air  Station  Oceana,  Virginia  Beach,  VA 
Naval  Air  Station  Patuxent  River,  Patuxent  River,  MD 
Naval  Aviation  Maintenance  Office,  Patuxent  River,  MD 
Naval  Air  Warfare  Center,  Aircraft  Division,  Indianapolis,  IN 
Naval  Air  Warfare  Center,  Patuxent  River,  MD 


Department  of  the  Air  Force 

Office  of  the  Deputy  Chief  of  Staff  (Logistics),  Washington,  DC 
Air  Combat  Command,  Langley  Air  Force  Base,  VA 
Dyess  Air  Force  Base,  Abilene,  TX 
Seymour  Johnson  Air  Force  Base,  Goldsboro,  NC 
Shaw  Air  Force  Base,  Sumter,  SC 

Air  Force  Materiel  Command,  Wright-Patterson  Air  Force  Base,  OH 
Oklahoma  City  Air  Logistics  Center,  Tinker  Air  Force  Base,  OK 
San  Antonio  Air  Logistics  Center,  San  Antonio,  TX 
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Contractors 

American  Airlines,  Tulsa,  OK 

General  Electric  Corporation,  Cincinnati,  OH 
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Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology 
Deputy  Under  Secretary  of  Defense  for  Logistics 
Director,  Defense  Logistics  Studies  Information  Exchange 
Under  Secretary  of  Defense  (Comptroller) 

Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Under  Secretary  of  Defense  for  Policy 

Assistant  to  the  Secretary  of  Defense  (Public  Affairs) 


Department  of  the  Army 

Assistant  Secretary  of  the  Army  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Navy 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Deputy  Chief  of  Staff  (Logistics) 

Auditor  General,  Department  of  the  Air  Force 


Defense  Organizations 

Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Logistics  Agency 
Director,  National  Security  Agency 

Inspector  General,  National  Security  Agency 
Inspector  General,  Defense  Intelligence  Agency 
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Non-Defense  Federal  Organizations  and  Individuals 

Office  of  Management  and  Budget 
General  Accounting  Office 

National  Security  and  International  Affairs  Division 
Technical  Information  Center 

Chairman  and  ranking  minority  member  of  each  of  the  following  congressional 
committees  and  subcommittees: 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  National  Security,  Committee  on  Appropriations 
House  Committee  on  Government  Reform  and  Oversight 
House  Subcommittee  on  National  Security,  International  Affairs,  and  Criminal 
Justice,  Committee  on  Government  Reform  and  Oversight 
House  Committee  on  National  Security 
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Part  HI  -  Management  Comments 


Deputy  Under  Secretary  of  Defense  for  Logistics 
Comments 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

3000  DEFENSE  PENTAGON 
WASHINGTON  DC  30301-3000 

1  4  JUN  1996 

MEMORANDUM  FOR  DEPARTMENT  OF  DEFENSE  INSPECTOR  GENERAL 

Subject:  Department  of  Defense  Inspector  General  "Audit  Report  on  Engine  Monitoring 
Svstems  for  Jet  Aircraft  Engines,"  Project  No.  5LB-0046 

This  is  the  Deputy  Under  Secretary  of  Defense  (Logistics)  response  to  subject 

audit. 

DUSD(L)  concurs  with  recommendation  (1).  Detailed  comments  are  provided  in 
the  attachment. 

John  F.  Phillips 
Deputy  Under  Secretary 
of  Defense  (Logistics) 


Attachment: 
As  stated 


acquisition  ano 

TECHNOLOGY 
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Deputy  Under  Secretary  of  Defense  for  Logistics  Comments 


RECOMMENDATION:  The  DoDIG  recommends  that  the  Deputy  Under  Secretary  of 
Defense  for  Logistics  provide  management  oversight  of  the  Joint  Advanced  Health  and 
Usage  Monitoring  System  program  to  ensure  that  the  Army  and  the  Navy  collect, 
process,  record,  and  effectively  use  data  generated  from  the  system  to  detect  and 
predict  engine,  component,  and  structural  failures  before  they  occur. 

•  DUSD(L)  RESPONSE:  Concur.  The  Joint  Advanced  Health  and  Usage 
Monitoring  System  (JAHUMS)  is  an  Advanced  Concept  Technology 
Demonstration  (ACTD).  It  will  involve  joint  Navy  and  Army  participation.  The 
Deputy  Under  Secretary'  of  Defense  (Advanced  Technology)  manages  and 
maintains  oversight  of  all  the  Department  of  Defense  ACTDs.  In  the  case  of  the 
JAHUMS  ACTD,  the  Deputy  Under  Secretary  of  Defense  (Logistics)  will  provide 
management  oversight  in  coordination  with  the  Deputy  Under  Secretary  of 
Defense  (Advanced  Technology').  The  JAHUMS  ACTD  is  scheduled  for  initiation 
in  Fiscal  Year  1997.  Specific  plans  are  dependent  upon  the  Fiscal  Year  1997 
budget  level  for  general  ACTD  funding.  Presently,  $16M  is  programmed  to 
demonstrate  the  JAHUMS  capability  in  a  number  of  test  aircraft.  General 
JAHUMS  ACTD  goals  will  be  to  demonstrate  the  following: 

<*  30%  reduction  in  Class  A  material  mishaps 

«  90%  probability’  of  detecting  incipient  mechanical  component  failures 

*  <1%  false  alarm  rate 

*  25%  reduction  in  maintenance  costs /flight  hour 

»  40%  reduction  in  airframe  vibration 

«  95%  accuracy  in  structural  loads  prediction 

The  goals  satisfy  the  Joint  Technology'  Development  Approach  (TDA)  Objectives. 
Specific  aircraft  model  goals  will  be  based  upon  available  maintenance  baseline 
performance. 
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DEPARTMENT  OF  THE  ARMY 
OFFICE  OF  THE  DEPUTY  CHIEF  OF  STAFF  POP  LOGISTICS 
$00  APWY  PENTAGON 
WASMMGTON.  DC  303100600 


0  3  JUN  1996 


DALC-SMV 


MEMORANDUM  THRU 


DEPUTY  CHIEF  OF  STAFF  FOR  LOGISTICS 


P.  QUEUE.  LTC.Qa.ADFCC 


ASSISTANT  SECRETARY  OlCsHE  ARMY  ( INSJEAbCgflONS ,  lOGI^WdcftOnlB 
ENVIRONMENT)  deputy  f  iSiStant 

(Logistics) 

FOR  INSPECTOR  GENERAL,  DEPARTMENT  CF  DEFENSE  (AUDITING*^*  <I4L> 


SUBJECT:  Audit  Report  on  Engine  Monitoring  Systems  for  Jet 

Aircraft  Engines  (Project  No.  5LB~ 0046)  ~~ INFORMATION  MEMORANDUM 

1.  This  is  in  response  to  USAAA  memorandum  of  8  April  1996. 
(Tab  A),  which  asked  ODCSLOG  to  respond  to  your  memorandum  of 
5  April  1996  (Enel  to  Tab  A)  .  Your  memorandum  requested  that 
ODCSLOG  review  the  findings,  recommendations,  or  estimated 
monetary  benefits  of  subject  report. 

2.  The  Army's  response  to  the  subject  draft  audit  is  at  Tab  B. 


±  tnc is 


WIMPK^  PYBUS 
Director  of  Maintenance 
Management 


YCSA 
CDR,  AMC 
CALC-ZXA 

z_M C  (AMCIR-A)  -  Concur,  Mr,  Xurzer/617-9025  {by  phone) 
A.TCCM  (AMSAT-3-L)  -  Concur,  Mr.  Huseman/ 693-3777  (by  phone) 


Department  of  the  Army  Comments 


Array  response  to  Audit  Report  on  Engine  Monitoring  Systems  for 
Jet  Aircraft  Engines  (Project  No.  5 LB-0046) 


The  following  paragraphs  list  the  Army  portions  of  the  DODIG 
recommendations  followed  by  the  response,  as  applicable: 

a.  Recommendation  2.  The  Commander,  Aviation  and  Troop 
remand  {ATCOM)  issue  guidance  requiring  that  performance 
specifications  be  developed  for  all  engine  monitoring  system 
(EMS)  under  development.  The  specifications  should  ensure  that 
the  systems  generate  the  data  needed  to  monitor,  diagnose,  and 
trend  the  condition  of  engines. 

b.  Army  Response.  Concur.  An  EMS,  as  a  part  of  the  T800 
engine  development  program,  and  the  Built-in-Test  subsystems  for 
the  Subsystem  Power  Unit  are  currently  being  developed.  Both  are 
for  the  RAH- 6 6  Comanche  and  both  are  being  developed  to 
performance  specifications  that  we  believe  will  ensure  that  the 
appropriate  data  are  generated  to  monitor  engine  condition, 
diagnose  faults,  and  trend  engine  performance. 

c.  Recommendation  3.  The  Commander,  ATCOM  selectively  test 
the  consistency  and  timeliness  of  EMS  data  being  reported  by 
maintenance  organizations.  Where  determined  necessary, 
additional  training  should  be  provided  to  maintenance  personnel 
on  timely  reporting  of  EMS  data. 

d.  Army  Comment.  Concur.  Action  has  been  initiated  to 
improve  data  reporting  for  the  T-700  engine.  DA  Form  2408-19-3 
data  were  compared  to  DA  Form  2410  installed  data  and 
discrepancies  noted.  On  29  Feb  96,  156  letters,  with  1024 
aircraft/aircraft  serial  number  combinations,  were  sent  out. 

To  date,  42  percent  of  the  organizations  have  responded  with 
clarifications  and/or  data  corrections.  This  process  will  be 
repeated  to  continually  monitor  and  improve  the  timeliness  and 
accuracy  of  data  reporting.  In  addition,  we  are  supporting  the 
development  ana  fielding  of  Unit  Level  Logistics  System-Aviation 
(ULLS-A)  .  We  have  submitted  requirements  to  the  Program  Manager, 
ULLS-A,  for  the  automated  electronic  transmission  of  DA  Fora 
2408-19-3  data  to  ATCCM.  When  implemented,  this  will  lessen  the 
burden  on  the  units  and  improve  accuracy  and  timeliness  of  data 
input.  ULLS-A.  is  currently  being  fielded  and  the  transmission  of 
data  to  ATCCM  should  begin  in  FY  98. 

e.  Recommendation  4.  The  Commander,  ATCOM  develop  guidance 
to  establish  useful  performance  measures  to  determine  the  coat 
and  operational  effectiveness  of  the  engine  monitoring  system 
programs  aireaay  in  existence  and  those  under  development. 

Special  emphasis  should  be  placed  on  criteria  for  determining 
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whether  the  EMS  reduces  unscheduled  engine  removals,  maintenance 
man-hours,  or  air  and  ground  aborts. 

f.  Army  Response.  Partially  concur. 

Concur  with  regards  to  the  system  under  development.  The 
ATCCM  Directorate  for  Maintenance  will  perform  an  analysis  of 
field  maintenance  oerformance  data  on  the  T800  EMS  in  order  to 
assess  value  added* by  the  diagnostic  system.  The  analysis  will 
require  the  engines  to  be  fielded  for  some  period  of  time  to 
develop  failure  diagnostic  history  and  will  be  contingent  upon 
funding  support  from  the  Aviation  Program  Executive 
Office/Comanche  Program  Manager. 

Nonconcur  with  regard  to  the  system  in  existence.  The  T700 
engine  series  history  recorder/counter  is  not  an  EMS  and  thus  an 
analysis  to  determine  whether  it  reduces  unscheduled  engine 
removals,  maintenance  man-hours,  or  air  and  ground  aborts  is  not 
feasible.  As  mentioned  in  the  report,  the  history  recorder  was 
not  designed  to  provide  diagnostic  and  trending  capabilities. 

2.  Additional  comments. 

a.  The  DODIG  states  that  existing  EMS  did  not  provide  all 
the  information  needed  to  diagnose,  monitor,  and  trend  the 
condition  of  the  engines.  It  should  be  noted  that  the  T700 
engine  series  history  recorder/counter  was  not  designed,  nor  was 
there  a  mission  need  statement,  for  that  purpose. 

b.  Reference  page  10,  second  paragraph.  The  Army  at  ATCQM 
does  not  use  the  history  recorder  data  to  control  the  maximum 
allowable  operating  time  between  overhaul  of  the  Apache  and 
Slackhawk  helicopters.  Both  the  aircraft  and  engine  are  "on- 
conaiticn"  and  the  responsibility  for  component  removal  on  life 
* imited  components  lies  solely  with  owning  units.  Only  in 
special  cases,  such  as  Safety-of-Flight,  does  ATCOM  direct 
removal  of  components. 

c.  Reference  page  10,  second  paragraph,  last  sentence. 

DODIG  states  "...the  data  were  furnished  to  General  Electric  to 
determine  useful  life  of  engine  components."  This  statement  is 
incorrect.  Useful  life  is  not  based  on  historical  data. 
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